Goto

Collaborating Authors

 chat assistant


Assessing Web Search Credibility and Response Groundedness in Chat Assistants

arXiv.org Artificial Intelligence

Chat assistants increasingly integrate web search functionality, enabling them to retrieve and cite external sources. While this promises more reliable answers, it also raises the risk of amplifying misinformation from low-credibility sources. In this paper, we introduce a novel methodology for evaluating assistants' web search behavior, focusing on source credibility and the groundedness of responses with respect to cited sources. Using 100 claims across five misinformation-prone topics, we assess GPT-4o, GPT-5, Perplexity, and Qwen Chat. Our findings reveal differences between the assistants, with Perplexity achieving the highest source credibility, whereas GPT-4o exhibits elevated citation of non-credibility sources on sensitive topics. This work provides the first systematic comparison of commonly used chat assistants for fact-checking behavior, offering a foundation for evaluating AI systems in high-stakes information environments.


LLM should think and action as a human

arXiv.org Artificial Intelligence

It is popular lately to train large language models to be used as chat assistants, but in the conversation between the user and the chat assistant, there are prompts, require multi-turns between the chat assistant and the user. However, there are a number of issues with the multi-turns conversation: The response of the chat assistant is prone to errors and can't help users achieve their goals, and as the number of conversation turns increases, the probability of errors will also increase; It is difficult for chat assistant to generate responses with different processes based on actual needs for the same prompt; Chat assistant require the use of tools, but the current approach is not elegant and efficient, and the number of tool calls is limited. The main reason for these issues is that large language models don't have the thinking ability as a human, lack the reasoning ability and planning ability, and lack the ability to execute plans. To solve these issues, we propose a thinking method based on a built-in chain of thought: In the multi-turns conversation, for each user prompt, the large language model thinks based on elements such as chat history, thinking context, action calls, memory and knowledge, makes detailed reasoning and planning, and actions according to the plan. We also explored how the large language model enhances thinking ability through this thinking method: Collect training datasets according to the thinking method and fine tune the large language model through supervised learning; Train a consistency reward model and use it as a reward function to fine tune the large language model using reinforcement learning, and the reinforced large language model outputs according to this way of thinking. Our experimental results show that the reasoning ability and planning ability of the large language model are enhanced, and the issues in the multi-turns conversation are solved.


LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory

arXiv.org Artificial Intelligence

Recent large language model (LLM)-driven chat assistant systems have integrated memory components to track user-assistant chat histories, enabling more accurate and personalized responses. However, their long-term memory capabilities in sustained interactions remain underexplored. This paper introduces LongMemEval, a comprehensive benchmark designed to evaluate five core long-term memory abilities of chat assistants: information extraction, multi-session reasoning, temporal reasoning, knowledge updates, and abstention. With 500 meticulously curated questions embedded within freely scalable user-assistant chat histories, LongMemEval presents a significant challenge to existing long-term memory systems, with commercial chat assistants and long-context LLMs showing 30% accuracy drop on memorizing information across sustained interactions. We then present a unified framework that breaks down the long-term memory design into four design choices across the indexing, retrieval, and reading stages. Built upon key experimental insights, we propose several memory designs including session decomposition for optimizing value granularity, fact-augmented key expansion for enhancing the index structure, and time-aware query expansion for refining the search scope. Experiment results show that these optimizations greatly improve both memory recall and downstream question answering on LongMemEval. Overall, our study provides valuable resources and guidance for advancing the long-term memory capabilities of LLM-based chat assistants, paving the way toward more personalized and reliable conversational AI.


Google's Pixel 9 could arrive with a sophisticated 'Pixie' AI assistant

Engadget

Google is creating a new, more sophisticated Android AI assistant called Pixie set to arrive with its Pixel 9 phone, according to a report from The Information. Based on the company's new Gemini large language model (LLM), it'll be able to perform "complex and multimodal tasks" like giving you directions to the nearest store to buy a product you photographed on your smartphone. The assistant will be exclusive to Google's Pixel devices and use data from Google products like Gmail and Maps. That would help it "evolve into a far more personalized version of the Google Assistant," the report states. It appears to be a separate product from Google's Assistant with Bard showed off at Made By Google in October.


ExpertPrompting: Instructing Large Language Models to be Distinguished Experts

arXiv.org Artificial Intelligence

The answering quality of an aligned large language model (LLM) can be drastically improved if treated with proper crafting of prompts. In this paper, we propose ExpertPrompting to elicit the potential of LLMs to answer as distinguished experts. We first utilize In-Context Learning to automatically synthesize detailed and customized descriptions of the expert identity for each specific instruction, and then ask LLMs to provide answer conditioned on such agent background. Based on this augmented prompting strategy, we produce a new set of instruction-following data using GPT-3.5, and train a competitive open-source chat assistant called ExpertLLaMA. We employ GPT4-based evaluation to show that 1) the expert data is of significantly higher quality than vanilla answers, and 2) ExpertLLaMA outperforms existing open-source opponents and achieves 96\% of the original ChatGPT's capability. All data and the ExpertLLaMA model will be made publicly available at \url{https://github.com/OFA-Sys/ExpertLLaMA}.


4 Ways to Benefit From Conversational Bots in 2020

#artificialintelligence

Customers love voice and chat assistants, the conversational interfaces that turn on the lights, help home chefs cook an egg to perfection, and make it easy for consumers to research and buy goods online. However, while customers are already building strong relationships with these conversational assistants, retailers are still learning how to best use conversational bots to drive engagement and strengthen their customer relationships. Nonetheless, these conversational assistants represent a fantastic opportunity for retailers to humanize their interactions with customers at scale, as long as it's done with proper understanding of what it takes to engage with customers and how to deploy voice and chat to drive growth and return in 2020. Conversational interfaces fall into two categories: voice and chat. Voice assistants are mediums that can be accessed through voice commands on a smart speaker or smartphone application. Examples include Google Home and Google Assistant, Amazon Alexa, Apple Siri, and Microsoft Cortana.


Are AI capabilities the force of the future, or is it emotional intelligence?

#artificialintelligence

Anywhere, at any time, the capabilities and competence of Artificial Intelligence (AI) has the potential to threaten an employee's role and an employer's style of working. Recent statistics surrounding the correlation between AI and employment in the Capgemini survey reveal that an increasing number of employers are turning to AI devices to heighten their company's impact. For instance, when discussing conversational AI chat bots, 76 percent of the organisations interviewed have seen quantifiable benefits from their voice and chat initiatives, while 58 percent claimed that these benefits met or exceeded their expectations. Compared to last year, the number of consumers using voice has shown a meaningful increase and the report emphasises that customers are increasingly preferring to use voice assistants throughout the consumer journey. To leverage their grasp of consumer appetites, the organisations appear to be trying to find the right balance between human and robotic interactions, but this still threatens the existence of human-based roles in their companies.


70% will swap store visits for voice assistants by 2022

#artificialintelligence

Chatbots and voice assistants have been derided for providing limited functionality and stilted conversations that frustrate consumers. Capgemini's survey indicates that voice assistants have come a long way in the past few years to improve customer satisfaction and are on course to save money for companies by automating a broader range of services now handled by humans. More than three-quarters of businesses (76%) have seen quantifiable benefits from voice or chat assistants, while 58% said that the benefits had met or exceeded their expectations, per Capgemini. Some businesses have been able to cut their customer service costs by more than 20% as consumers turned to voice assistants instead of humans, but that benefit hasn't translated into broader adoption among companies. Fewer than 50% of the top 100 businesses in the automotive, consumer products and retail, and banking and insurance industries have voice assistants or chatbots, Capgemini's survey found.


Smart Talk

#artificialintelligence

Conversational assistants are here to stay, making everything from boiling an egg to making a payment that much easier. And consumers expect more of them day by day. If they meet these growing expectations, conversational assistants are in a position to transform the customer experience landscape. But do organizations have the customer centricity and organizational capabilities necessary to deploy these technologies successfully? In the new report from the Capgemini Research Institute, Smart Talk: How organizations and consumers are embracing voice and chat assistants,we talked to over 12,000 consumers who've used and continue to use voice and/or chat assistants and to 1,000 executives from consumer products and retail, financial services, and automotive, including pure-play digital players.


AI virtual assistants making banking easier for everyday consumers

#artificialintelligence

The rise of mobile banking in the last few years has coincided with the rise of Artificial Intelligence (AI) becoming more capable and pervasive in a variety of fields and applications. Although unrelated, the parallel growth in both the sectors has, of course, led to a confluence of the two, as more and more banking and finance institutions explore AI applications to improve their offerings and services. While enterprise-solutions are many – from more efficient data analytics to improved back-end workflows through automation – quite a few organisations have also explored consumer-facing applications of AI. One of the most widely adopted applications has been the consumer chat-assistant. Simply put, a chat assistant (colloquially called a chatbot) is an AI or computer programme that conducts conversations meant to replicate human conversation via auditory or textual means.